A computational approach to gestural interactions of the upper limb on planar surfaces

There are many compelling reasons for proposing new gestural interactions: one might want to use a novel sensor that affords access to data that couldn’t be previously captured, or transpose a well-known task into a different unexplored scenario. After an initial design phase, the creation, optimisation or understanding of new interactions remains, however, a challenge. Models have been used to foresee interaction properties: Fitts’ law, for example, accurately predicts movement time in pointing and steering tasks. But what happens when no existing models apply? The core assertion to this work is that a computational approach provides frameworks and associated tools that are needed to model such interactions. This is supported through three research projects, in which discriminative models are used to enable interactions, optimisation is included as an integral part of their design and reinforcement learning is used to explore motions users produce in such interactions

Gesture Typing on Virtual Tabletop: Effect of Input Dimensions on Performance

The association of tabletop interaction with gesture typing presents interaction potential for situationally or physically impaired users. In this work, we use depth cameras to create touch surfaces on regular tabletops. We describe our prototype system and report on a supervised learning approach to fingertips touch classification. We follow with a gesture typing study that compares our system with a control tablet scenario and explore the influence of input size and aspect ratio of the virtual surface on the text input performance. We show that novice users perform with the same error rate at half the input rate with our system as compared to the control condition, that an input size between A5 and A4 present the best tradeoff between performance and user preference and that users' indirect tracking ability seems to be the overall performance limiting factor

Using models of baseline gameplay to design for physical rehabilitation

Modified digital games manage to drive motivation in repetitive exercises needed for motor rehabilitation, however designing modifications that satisfy both rehabilitation and engagement goals is challenging. We present a method wherein a statistical model of baseline gameplay identifies design configurations that emulate behaviours compatible with unmodified play. We illustrate this approach through a case study involving upper limb rehabilitation with a custom controller for a Pac-Man game. A participatory design workshop with occupational therapists defined two interaction parameters for gameplay and rehabilitation adjustments. The parameters' effect on the interaction was measured experimentally with 12 participants. We show that a low-latency model, using both user input behaviour and internal game state, identifies values for interaction parameters that reproduce baseline gameplay under degraded control. We discuss how this method can be applied to systematically balance gamification problems involving trade-offs between physical requirements and subjectively engaging experiences.Comment: 19 pages, 10 figure

Describing movement learning using metric learning

International audienceAnalysing movement learning can rely on human evaluation, e.g. annotating video recordings, or on computing means in applying metrics on behavioural data. However, it remains challenging to relate human perception of movement similarity to computational measures that aim at modelling such similarity. In this paper, we propose a metric learning method bridging the gap between human ratings of movement similarity in a motor learning task and computational metric evaluation on the same task. It applies metric learning on a Dynamic Time Warping algorithm to derive an optimal set of movement features that best explain human ratings. We evaluated this method on an existing movement dataset, which comprises videos of participants practising a complex gesture sequence toward a target template, as well as the collected data that describes the movements. We show that it is possible to establish a linear relationship between human ratings and our learned computational metric. This learned metric can be used to describe the most salient temporal moments implicitly used by annotators, as well as movement parameters that correlate with motor improvements in the dataset. We conclude with possibilities to generalise this method for designing computational tools dedicated to movement annotation and evaluation of skill learning

Describing movement learning using metric learning

International audienceAnalysing movement learning can rely on human evaluation, e.g. annotating video recordings, or on computing means in applying metrics on behavioural data. However, it remains challenging to relate human perception of movement similarity to computational measures that aim at modelling such similarity. In this paper, we propose a metric learning method bridging the gap between human ratings of movement similarity in a motor learning task and computational metric evaluation on the same task. It applies metric learning on a Dynamic Time Warping algorithm to derive an optimal set of movement features that best explain human ratings. We evaluated this method on an existing movement dataset, which comprises videos of participants practising a complex gesture sequence toward a target template, as well as the collected data that describes the movements. We show that it is possible to establish a linear relationship between human ratings and our learned computational metric. This learned metric can be used to describe the most salient temporal moments implicitly used by annotators, as well as movement parameters that correlate with motor improvements in the dataset. We conclude with possibilities to generalise this method for designing computational tools dedicated to movement annotation and evaluation of skill learning

Insulin-like growth factor (IGF-1) receptors during human lung development

International audienc

Adaptive optics light-sheet microscopy based on direct wavefront sensing without any guide star

International audienceWe propose an Adaptive Optics Light-Sheet Fluorescence Microscope (AO-LSFM) for closed-loop aberrations correction at the emission path, providing intrinsic instrumental simplicity and high accuracy when compared to previously reported schemes. The approach is based on direct wavefront sensing i.e. not on time-consuming iterative algorithms, and does not require the use of any guide star thus reducing instrumental complexity and/or sample preparation constraints. The design is based on a modified Shack-Hartmann wavefront sensor providing compatibility with extended sources such as images from optical sectioning microscopes. We report an AO-LSFM setup based on such sensor including characterization of the sensor performance, and demonstrate for the first time significant contrast improvement on neuronal structures of the ex-vivo adult drosophila brain in depth

Dynamics of mitochondrial membranes under photo-oxidative stress with high spatiotemporal resolution

International audienceIn our study, we harnessed an original Enhanced Speed Structured Illumination Microscopy (Fast-SIM) imaging setup to explore the dynamics of mitochondrial and inner membrane ultrastructure under specific photo-oxidation stress induced by Chlorin-e6 and light irradiation. Notably, our Fast-SIM system allowed us to observe and quantify a distinct remodeling and shortening of the mitochondrial structure after 60–80 s of irradiation. These changes were accompanied by fusion events of adjacent inner membrane cristae and global swelling of the organelle. Preceding these alterations, a larger sequence was characterized by heightened dynamics within the mitochondrial network, featuring events such as mitochondrial fission, rapid formation of tubular prolongations, and fluctuations in cristae structure. Our findings provide compelling evidence that, among enhanced-resolution microscopy techniques, Fast-SIM emerges as the most suitable approach for non-invasive dynamic studies of mitochondrial structure in living cells. For the first time, this approach allows quantitative and qualitative characterization of successive steps in the photo-induced oxidation process with sufficient spatial and temporal resolution